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1. INTRODUCTION 

Drowsiness and fatigue are one of the significant factors that affect road safety and cause serious 
accidents, casualties, and economic losses [1]. According to national highway traffic safety administration 
(NHTSA) report in the United States, between 2011 and 2015 about 2.4% of 153,297 road accidents, more 
than 1.25 million deaths annually, and 20-50 million people injured or disabled are caused by drowsy 
driving; In addition to the high cost which can be reached to $518 billion. In India, according to Ministry of 
Road transport and highways transport research wing report, there were about 449,002 accidents out of this 
number of data 33.65% was the mortality rate in 2019. While in Europe, in 2019 the mortality rate was 19% 
due to road accident. Statistics predicts that annually about 76,000 injuries and 1,200 deaths can caused 
because of fatigue and drowsiness through driving. As a result, by 2030 road traffics will become the fifth 
cause of death. In air traffic, Drowsiness and fatigue become a serious issue. According to recent surveys, 
long flight and busy schedule of the pilots lead them to take a short snap or snooze through their flight. 
Therefore, airline industry must ensure that their crew is a wake periodically or it will be a major problem. 
There are numerous factors that contribute to driver’s fatigue such as: lack of rest, a long ride, wakefulness, 
consumption of alcohol, and mental stress. Those factors may lead to serious disaster if action does not take 
on time. Moreover, previous transportation system was not provided with tools or techniques to control those 
factors on roads [2]-[4]. 

Due to the dangerous accidents that fatigue poses on the roads, researchers developed different types 
of methods to detect driver’s drowsiness. Despite the large number of methods conducted in this field and the 
numerous technologies available for driver’s drowsiness detection, it is still unclear which one is the most 
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appropriate depending on cars and driver’s conditions; In addition, what enhancement can be done in the 
future about this topic. To conduct the best results from this review paper, searching keywords to collect the 
relevant information have been proposed. They focus on their search on journals and conferences that have 
high reputations. In this review paper, i) we classify the drowsiness detection techniques into three main 
categories and discuss for each category its pros and cons in addition to its limitations, ii) we review the 
machine learning classification techniques that are used to detect drowsiness, and iii) we recommend 
combining audio as a new feature with the three extracted features-based techniques to enhance the detection 
of drivers’ drowsiness. 

This paper is divided into six main parts. The second part describes the way that existing studies 
have been collected. Next following drowsiness detection techniques are presented in detail. Then presents a 
comparative study of drowsiness detection techniques using machine learning classification techniques. 
While the last part discusses the comparative analysis of machine learning classification for drowsiness 
detection techniques. Finally, discussed the conclusion of this study. 


2. RESEARCH METHOD 

The goal of this systematic review is to illustrate the workflow of machine learning to detect drivers’ 
drowsiness, highlights the supervised machine learning techniques that have been successfully employed in 
real-world to detect drivers’ drowsiness. In addition to define the pros and cons of each technique and 
provide suggestions to enhance those detection techniques. Therefore, the existing research papers relevant to 
drowsiness detection techniques have been collected from high reputed journals’ search engines: IEEE 
Explore, arrhythmogenic cardiomyopathy (ACM), and Google Scholar. Based on the keyword search, about 
2,500 existing papers were retrieved in primary search, as illustrated in Table 1. To make the selection and 
retrieving method more accurate, commands that compose of logic expressions and search keywords have 
been applied on abstract and title as shown in Table 2. As a result, the number of collected papers has been 
reduced to 48. Finally, we divided the 48 papers into three main categories depending on the extracted 
features that play important role in supervised drivers’ drowsiness machine learning detection techniques, 
those categories are: 
a. Vehicle features-based techniques, 
b. Behavioral features-based techniques, and physiological features-based techniques. 


Table 1. Description of search words 


Search of word Set of keywords 

Drow* Drowsiness, Drowsiness detection, Drowsiness technique, drowsy driver 

Fatigu* Fatigues driver, Fatigue detection, Fatigue detection technique 

Biological* Biological parameters, Biological measures 
Physiological* Physiological parameters, Physiological measures 
Car* Car measures, car movements 

Vehicle Vehicle measures, Vehicle movements 
Re Behavioral* Behavioral parameters, Behavioral measures 

Classif* Classifier, Classification technique, Classification Methods 


Table 2. Search keywords commands 


Commands Description 

Command! (Biological*OR physiological*) AND (Drows*) OR (Fatigu*) 
Command2 (Vehicle*OR Car*) AND ((Drows*) OR (Fatigu*)) 
Command3 (Behavioral*) AND ((Drows*) OR (Fatigu*)) 
Command4 (Behavioral*) AND (EOG*) AND ((Drows*) OR (Fatigu*)) 
Command5 (Classif**) AND ((Drows*) OR (Fatigu*)) 
Command6 (Emotion*) AND ((Drows*) OR (Fatigu*)) 


3. GENERAL WORKFLOW FOR DETECTING DRIVER DROWSINESS USING SUPERVISED 
MACHINE LEARNING TECHNIQUES 
Supervised machine learning techniques are a common type of machine learning that used for 
predicting, forecasting or classification tasks. They are named supervised learning because the learning 
algorithm is carried out using label data. Figure 1 demonstrates the general steps for driver drowsiness 
detection using supervised machine learning techniques. 
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Figure 1. Supervised driver drowsiness detection techniques workflow 


3.1. Data acquisition 
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The first step in any supervised machine learning technique is gathering or collecting data. The size 
of the data collecting, and the quality play a crucial role to build an accurate model. From the previous 
research papers, some authors use datasets that are published, validated, and accessed for free to detect 
drivers’ drowsiness. For example, driver drowsiness detection (DDD) dataset [5], closed eyes in the wild 
(CEW) [6], National Tsing Hua University (NTHU) DDD dataset [7], [8], DROZY dataset [9], SleepEYE I 
project (SleepEYE) dataset [10], and yawning detection dataset (YawDD) [11], [12]. While others create 
their own dataset and validate it using different machine learning techniques as in [13]-[15]. Table 3 


illustrates the types of datasets used in DDD. 


Table 3. Types of datasets in DDD 


Dataset Name Vehicle Features Behavioral Features Physiological Features 


DDD [5] 
CEW [6] 
YawDD [11], [12] 
SleepEYE [10] v 
NTHU-DDD [7], [8] 
DROZY [9] 


VSR KKS 


3.2. Data cleaning and preprocessing 


Data that has been collected cannot be directly fed into machine learning workflow in its current 
format. Therefore, raw data must clean and transform to a format that can be understandable by the machine. 
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Data cleaning referred to the process of removing unnecessary or corrupted piece of data within dataset and 
fixing the errors that occurs in the data to assure reliable algorithms and outcomes. In DDD techniques, raw 
data can be in different formats. It can be gathered from sensors, images, and videos. For data that extracted 
from sensors, those data are in numeric format, to clean this type of data we must use the following steps: fill 
the missing values and remove noisy and null values. To clean image data, there are general steps that should 
be followed to clean data, those steps are cropping, filtering, rotating, or flipping images. For video 
preprocessing, video data should be split into frames, each frame treated as an image that follows the image 
preprocessing steps [16], [17]. 
3.3. Feature extraction 

Feature is a fixed number of characteristics, which might be binary, categorical, or continuous that 
used to represent data. The terms “feature” represents input variable or attribute. Finding a good data 
representation is domain specific and dependent on the measurements that are available [17]. We classify 
supervised machine learning techniques depending on the extracted features into three main categories. 


Vehicular features-based techniques, behavioral features-based techniques, and _physiological-based 
techniques. Figure 2 demonstrates the classification of drowsiness detection techniques. 
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Figure 2. The basic structure of sleep detection techniques 
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3.3.1. Vehicular features-based technique 

This type of features is extracted depending on the sensors that are installed in the steering wheel. 
Sensors are used to detect drivers steering behaviors by monitoring and measuring the steering movements. 
steering wheel angle (SWA), steering wheel velocity (SWV), and approximate entropy (AP) represent this 
type of features. McDonald et al. [18] apply the decision-tree (DT) after extracting the ApEn of SWA 
features to detect driver’s drowsiness and gain an accuracy of 79%. On real time Li et al. [19] use the SWA 
features and AP features. They input the features into binary classifier. The results show that the proposed 
system has an accuracy of 78.01%. The work in [20] apply the random forest classifier (RFC) after extracting 
the SWA features to detect driver’s drowsiness and gain an accuracy of 82%. For SWA Chai et al. [21] select 
four types of (SWA) features and fed them into three types of classifiers, the results show that using SWA 
features with multilevel ordered logit (MOL) classifier has better accuracy than the other two classifiers. 
Finally, Arefnezhad et al. [22] extracts vehicular features that is related to SWA and SWV; it uses the 
adaptive neuro-fuzzy feature selection to select the optimal features before inputting them to the support 
vector machine (SVM) classifier. The proposed model achieves 98.12 accuracy. Table 4 briefly summarized 
the vehicular feature-based technique. Techniques that use this type of features are more reliable. However, 
they face some limitations. As this technology has advantages is non-intrusive and disadvantage that effected 
by geometric characteristics of roads, unreliable [3], [4], [23]. Table 4 illustrates a brief utilization of vehicle 
techniques. 


3.3.2. Physiological features-based techniques 
Physiological techniques are based on features that extracted from driver’s physical condition. 


Features such as: (heart, pulse, breath, and respiratory) rate; in addition to body temperature are represent this 
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type of technique. Many previous studies use those features to detect drivers’ drowsiness. While in [24], 
authors was developed a model that enhanced the electrocardiogram (ECG) and heart rate variability (HRV) 
feature and use the artificial neural network (ANN) to detect drivers’ drowsiness. The accuracy rate for the 
technique in the detection of drowsiness was 90%. Li et al. [25] developed a model that enhanced the 
electroencephalogram (EEG) pulse and use the SVM based posterior probabilistic model (SVMPPM) to 
detect drivers’ drowsiness. The accuracy rate for the technique in the detection of drowsiness was 91.25%. 
The research in [26] was used Bayesian-copula discriminant classifier (BCDC) based on the physiological 
features extracted from EEG signals. The accuracy rate for the proposed technique for the detection of 
drowsiness was 94.30%. The work in [27] proposed an algorithm to detect slow eye movement; It starts by 
extracting the EEG features from occipital O2 signal, then the extracted features are added to the horizontal 
electro-oculogram (HEOG) signal features set, after that maximum relevance and minimum redundancy 
(mRMR) feature selection method is applied to select the optimal feature which fed into the SVM learning. 
As a result, the accuracy has been increased by 1.4% when using the extracted EEG power features. Tabal 
and Dela Cruz [28] developed a model that enhanced the electrooculogram (EOG) blink pulse feature and use 
the nearest neighbor algorithm classifier to detect drivers’ drowsiness. The accuracy rate for the technique in 
the detection of drowsiness was 87.14%. The work in [29] applied four types of classifiers (K-NN, SVM, 
CBR, and RF) to detect driver drowsiness after extracting the EEG, EOG features. Results showed that SVM 
achieved and accuracy of 93% for binary classification. While in [30] EEG signals have been extracted and 
divided into multiple frequency bands using wavelet packet transformation to determine if the functional 
brain network transitions from the warning state to the sleepy state. Two types of feature selection algorithms 
have been used synchronization likelihood (SL) and minimum spanning tree (MST). The selected features 
are input to four types of classifiers (SVM, K-NN, logistic regression (LR), and DT). The results show that 
K-nearest neighbors (K-NN) has accuracy of 98.6% when SL and MST are used together as a feature 
selection algorithm. The research in [13] applied a model that extracted the EOG signals features, they used a 
K-NN as a machine learning trainer and achieved and accuracy of 95.34%. Table 5 illustrates a brief 
utilization of physiological signs. 


Table 4. Summary of artworks utilizing vehicle techniques 


Ref Year Vehicle features Supervised Machine Learning Accuracy 
[18] 2012 SWA Random forest algorithm 79.00% 
[19] 2017 SWA+AP Binary classifier 78.01% 
[31] 2017 ApEn of SWA Decision-tree 82.00% 
[21] 2018 SWA MOL 72.92% 
[22] 2019 SWA+SWV Adaptive neuro-fuzzy feature selection with SVM classifier 98.12% 


Table 5. Summary of artworks utilizing physiological signs 


Ref Year Physiological Features Supervised Machine Learning Accuracy 
[24] 2011 ECG, HRV ANN 90% 
[25] 2015 EEG SVM based posterior probabilistic model (SVMPPM) 91.25% 
[26] 2016 EEG signal BCDC 94.30 
[27] 2016 EEG signal SVM 1.4% 
[28] 2017 EOG signal Nearest Neighbor 87.14% 
[29] 2018 EEG+EOG SVM 93% 
[30] 2018 EEG signals K-NN 98.6% 
[13] 2019 EOG signals K-NN 95.345% 


3.3.3. Behavioral features-based technique 

Behavioral techniques are non-invasive methods that depend on image processing to detect drivers’ 
behavioral features captured from the camera installed in the vehicle to track the driver’s actions and alert its. 
Features that are related to eyes, mouth, and head represent this type of technique. Saradadevi and Bajaj [31] 
suggested a methodology for detecting and alerting tiredness at an earlier stage and applied the SVM 
classifier on the extracted behavioral features that are associated with the mouth and yawing. Results show an 
improvement in accuracy of 83.3% on average. The work in [32] extracts eye index (EI), pupil activity (PA), 
and head poses (HP) to detect driver drowsiness, an SVM classifier has been used and record 94.84% 
accuracy. Head movement and eye closeness duration features have been extracted in [33] and fed into 
wavelet network classifier that based on neural network to detect drowsiness. The model achieved accuracy 
of 80%. Research [34] extracts mouth regions features based on circular Hough transforms and using a SVM 
as a classifier to detect drivers’ drowsiness. It scores an accuracy of 98%. In [35] behavioral features that 
related to eyes, mouth and face are extracted using the Viola-jones algorithm and cascade classifier. Results 
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show that the model has an accuracy of 90%. Pauly and Sankar [36] calculates the percentage of eye closures 
(PERCLOS) feature after detecting eye blinking, then the SVM are used, and the accuracy of the system was 
91.6%. Kumar and Patra [37] presents a technique that based on facial landmarks (FL) to calculate the eye 
aspect ratio (EAR), mouth opening ratio (MOR), and nose length ratio (NLR) and uses the SVM to detect 
driver’s drowsiness. The accuracy that the system achieved was 95.8%. Oliveira et al. [38] uses supervised 
machine learning classification (SVM, RF, ANN, gradient boosting tree (GBT), and KNN) On the features 
extracted from the video (head movement, eyelid opening, pupil diameter, and angle of the eye gaze). The 
results show that the classifiers record an accuracy of 89.15%. 

While Mehta et al. [39] present a model that can detect FL, measure the EAR and the eye closure 
ratio (ECR) of the driver based on adaptive thresholds. Random forest (RF) classifier achieved an accuracy 
of 84% to detect drivers’ drowsiness. Another work in [40] extracted the PERCLOS feature, eye blinking 
frequency (EBF), and maximum closure duration (MCD). All features are fed into different types of 
classifiers: (SVM, KNN, ANN, and LR). The results show that the KNN has an accuracy of 72.25% better 
than other classifiers. Costa et al. [41] presents a technique based on the eye and head tracker dataset (SEP), 
and uses the SVM and decision tree (DT) to detect driver’s drowsiness. The results show that the DT has an 
accuracy of 93% better than SVM. The work in [42] extract 35 behavioral features that related to eye 
blinking (EB) and head movement, and then use a feature selection method to select the optimal feature set 
that input to K-NN classifier. The results show that the model record accuracy of 84.2%. Zhang et al. [43] 
extracts five types of features that categorized as behavioral features and use AdaBoost classifier. Features 
are PERCLOS, MCD, blink frequency (BF), average opening level (AOL), opening and closing velocity 
(OV) capacitance-voltage (CV). As this technology has advantages is non-intrusive easy to use, and its 
disadvantages are affected by illumination conditions [3], [4]. Table 6 illustrates a brief utilization of 
behavioral techniques. 


Table 6. Summary of artworks utilizing behavioral techniques 


Ref Year Behaviors features Classification method Accuracy 
[31 2008 Yawing SVM 83.30% 
[32 2013 HP+PC+EI SVM 94.84% 
[33 2014 Head Movement+Eye Closeness Duration WN 80.00% 
[34 2014 Yawing SVM 98.00% 
[35 2015 EBR+Yawing Binary Cascade 90.00% 
[36 2015 PERCLOS SVM 91.60% 
[37 2018 Eye Aspect RatiotMouth Opening Ratio+Nose Length Ratio SVM 95.80% 
[38 2018 EOG signal+Head casings ens: ia diametert+Eye gaze SVM+RF+GBT 89.15% 
[39 2019 Facial Landmarks+eye aspect ratio (EAR)+eye closure ratio (ECR) RF 84.00% 
[40 2019 PERCLOS+EBF+MCD K-NN 72.25% 
[41 2019 Eye+Head SVM 93.00% 
[42 2020 EB+Head Movement K-NN 84.20% 
[43 2019 PERCLOS+BF+MCD+AOL+OV+CV AdaBoost 86.00% 


3.3.4. Hybrid featured-based technique 

Some authors have merge two or more features from different types to diagnose driver drowsiness 
and improve the overall accuracy of the model in a way that reduces limitations of one technique by using the 
features of the others. Another work [44] combines physiological, behavioral, and vehicular features such as: 
blood pressure, heart rate, temperature, and PERCLOS. Fuzzy Bayesian has been employed to detect drivers’ 
fatigue and record accuracy between 94% to 99%. Samiee et al. [45] extracts behavioral features and 
vehicular features for detecting drivers’ drowsiness. By using neural network classifier, the model records an 
accuracy of 87.78% at worse case. Table 7 illustrates a brief utilization of hybrid techniques. The research 
[46] has implemented combined non-contact approaches based on the voice and vision of somnolence 
detection systems. By calculating PERCLOS and using voice as a feature, SVM classifier an accuracy of 
98.6%. In Wang and Xu [47] extracts behavioral features and vehicular features and use MOL and ANN 
classifiers, results show that MOL has better accuracy than ANN classifier. 

Emotional stimulus based on pressure was introduced by [48] as critical feelings in addition to 
electrodermal activity (EDA) features. SVM classifier has been applied to detect drivers’ drowsiness and 
gain an accuracy of 85%. The study in [49] provided a novel technique for detecting drivers’ drowsiness 
through emotion that uses EEG data to identify human emotions using a two-tier classifier that combines the 
KNN and NN classifiers. The results show that the emotion recognition is a better choice in the current trend. 
The combined KNN-NN classifiers produces maximum classification accuracy of 97.10%. To detect drivers’ 
drowsiness, Li et al. [50] exploit the SWA features and the Yaw Angles feature. Both vehicular features and 
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behavioral features are used as input to the multi-level back propagation neural network classifier. 
Experiment show that the proposed system has achieved an accuracy of 88.02%. An ANNs classifier was 
applied on a combinations of vehicular, behavioral, and physiological features in [51]. The results show that 
the following features eyelid closure, gaze, and head movements have better performance with accuracy 
between 87% to 98%. Zhang [52] introduced the five basic emotions (entertainment, frustration, indignation, 
fear, and disgust); three psychophysiological signals were recorded: tEMG, EDA, and ECG. SVM were used 
with an accuracy of about 97.98%. Fernandes [53] uses supervised machine learning classification (SVM, 
RF, KNN, and bagging classifier (BC)) on the SWA and ECG. The higher result shows that the SVM 
classifier record accuracy of 94%. For example [54] use EEG and ECG features with the behavioral features 
of a driver to recognize drivers’ fatigue. The results show that the accuracy will be 78.7% when only the 
behavioral features are used while it increased to 89.8% when both physiological and behavioral features are 
used together in the RF classifier. Table 7 illustrates a brief utilization of hybrid methods. 


Table 7. Summary of artworks utilizing hybrid methods 


Ref Year Hybrid features Classification method Accuracy 
[44 2012 blood pressure+heart rate+temperature+PERCLOS Fuzzy Bayesian 94% to 99% 
[45 2014 Eye status+lateral position+tSWA Weighted output of three 87.78% 
trained neural networks 
[46 2015 Voice+PERCLOS SVM 98.60% 
[47 2015 PERCLOS, Pupil, BF, SWM Re, LP stdev MOL 88.60% 
[48 2016 emotional stimulation, EDA signals SVM 85.00% 
[49 2017 EEG signals for emotion recognition K-NN+NN 97.10% 
[50 2017 SWA+YA Neural Network 88.02% 
[51 2019 eyelid closure+gazethead movement ANN 87% to 98% 
[52 2019 essential feeling (entertainment, anger, outrage, dread, and SVM 97.98% 
disgust), (EMG), (EDA), and (ECG) 
[53 2019 SWA+ECG signals SVM 94.00% 
[54 2020 (Electroencephalogram and electrocardiogram) signals and RF 78.70% 
face 


4. LEARNING PROCESS 

In this step, extracted features must be input to the classifier, and then results are validated. 
Choosing the suitable classifier is a crucial step because it represents a key role in determining the accuracy 
of the prediction [55]. Therefore, knowing the pros and cons of each classifier can help in selecting the 
suitable classifier precisely. Machine learning classification approaches such as SVM, human visual system 
(HVS), Naive Bayes (NB), and K-NN represent the most discriminatory and appropriate classifiers’ 
techniques [56]—[58]. Table 8 illustrated the pros and cons of machine learning classifiers that help in 
detecting drivers’ drowsiness. 


Table 8. Machine learning classification approaches (pros and cons) [59]—[65] 


Classifier Name Pros Cons 
SVM - Works well when the margin - Jt does not work well when the dataset is large because it 
separator is clear, and the number of consumes more time in training data. 
dimensions exceeds the number of - Its performance will be less when the dataset has a lot of 
samples. noise. 


- Provide memory efficiency when 
works in high dimension spaces. 
K-NN - Easy to implement, especially for - Performance less when dataset is large and has noise on it. 
multi class classification. - It required that input features must be homogeneous. 
- Training step is not required. - 
- Accuracy is not affected when new 


data is added. 
Fuzzy Classifier - Simple, justifiable, cost low, easy to - Stability is not assured, trial and error are used for 
implement, and has intuitive design. optimization, and different tuning parameters are used. 
RF - It works well with nonlinear data. - Performance less when data is linear, and features are 

- Run efficiency when dataset is sparse. 
large. - It works slowly in training step. 

- It cannot affect by overfitting. 

ANN - A great ability to predict. - It takes unknown time to process. 
- Ideal for non-mathematical models. - It requires a large quantity of data for quality predictions 


- The Capacity to extract meaning 
from complex or imprecise 
knowledge 
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5. COMPARATIVE ANALYSIS OF DRIVER DETECTION TECHNIQUE AND DISCUSSION 

From the reviewed research paper, we compare the techniques from two points of view: i) the 
featured based techniques and ii) the type of the classifier that has been used by the technique. For the 
featured based technique point, it was noticed that vehicular features can give good results if it is not affected 
by the road circumstances, while behavioral features-based technique are easy to use but their performance 
will be reduced in poor lightening. The physiological-based techniques are efficient and reliable, but their 
intrusive nature required to be resolved. Hybrid features based technique can help in reducing the limitations 
of one feature by using other features from different type to increase the accuracy of the technique. From the 
classifier point of view, the SVM classifier is the most used one that gives good results either by using each 
feature separately or by using hybrid features. While KNN gives better results with physiological features; 
Moreover, applying large dataset with a good feature selection method can improve the accuracy of the 
classifier. 


6. CONCLUSION 

This review paper aims to shed light on the existing studies of driver drowsiness techniques, 
especially those techniques that use supervised machine learning. In this paper we give a brief information 
about dataset, workflow of supervised machine learning, features, classification techniques that used for 
detecting driver drowsiness. Driver detection techniques have been classified into four categories 
(physiological, vehicle, behavioral and hybrid) depending on the type of features that have been extracted. 
Comparative analysis showed that no one of the techniques can give high accuracy if they work lonely. 
Physiological features-based techniques give accuracy better than other techniques, but they suffer from 
intrusive. Therefore, hybrid features based techniques represents better solution to reduce the limitation of 
one technique by using the other ones. Most popular supervised machine learning techniques have been 
presented with their pros and cons while a comparative study show different results of accuracy with 
different situations. However, results for SVM classifier give better results with different types of features. 
Finally, we recommend combining audio as a new feature with the three extracted features-based techniques 
to enhance the detection of drivers' drowsiness. 
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